Europaisches Patentamt 
European Patent Office 
Office europeen des brevets 




© Publication number: 



0 534 350 A2 



@ Application number: 92116123.8 
© Date of filing: 21.09.92 



EUROPEAN PATENT APPLICATION 

© mt CIA H04N 7/137 



© Priority: 23.09.91 US 764187 


© Applicant: MATSUSHITA ELECTRIC 


INDUSTRIAL CO., LTD 


© Date of publication of application: 


1006, Oaza Kadoma, Kadoma-shi 


31.03.93 Bulletin 93/13 


Osaka 571 (JP) 


© Designated Contracting States: 


@ Inventor: Lynch, William E. 


DE FR GB NL 


27 Madison Street 




Princeton, NJ 08540(US) 




© Representative: Marx, Lothar, Dr. et al 




Patentanwalte Schwabe, Sandmair, Marx 




Stuntzstrasse 16 




W-8000 Munchen 80 (DE) 



© Derivation and use of motion vectors in a differential pulse code modulation system. 



© A differential pulse code modulation system hav- 
ing bidirectional motion estimation for blocks in B 
frames that is derived from the motion vector of a 



block in an anchor frame whose projection along its 
motion vector has the most overlap with the block in 
the B frame for which a motion estimation is sought. 
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Derivation and use of motion vectors in a differential poise code modwDation 
system 

Description of EP0534350 
BACKGROUND OF THE INVENTION 

In a Differential Pulse Code Modulation (DPCM) system referred to as Independent Search IS .that has 
been suqqested by the Moving Picture Experts Group and described in articles 1, 2 and 3, identified at the 
end of the specification that are hereby incorporated by reference, three types of frames are used as 
illustrated in Fig 1 . The letters indicate the type of frame, and the numerical subscripts indicate he order 
of occurrence of the frames at the input to the system. 10 and 19 are called intraframes Each of these 
intraframes is encoded by using only pixels within the frame. The encoded frames are then transmitted. 

A predicted frame such as P3 is encoded from 10 in the following manner. P3 is partitioned into identical 
blocks e g. having eight pixels in a side, that are called motion blocks. For each motion block in F»3 a 
search is made in the 10 frame for a block of the same shape and size that matches the motion block. 
When the matching block is located, its position relative to the motion block is indicated by a motion vector 
(mv) representing the number of pixels left or right and the number of pixels up or down from the mot.on 
block to the matching block. These motion vectors are transmitted. Also, the differences between the 
motion blocks and corresponding matching blocks are formed. This is called residue, and it is encoded 
and transmitted. At the decoder, the 10 frame is decoded and the predicted frame P3 is .formed by 
displacing appropriate matching blocks in 10 using the motion vectors/The residue for the various blocks is 
then added The frame P6 is encoded in like manner from P3. When a number of P frames have been 
encoded from preceding P frames in this manner, the accumulative error may 'become so great that 
another I frame is sent as described, and a new series of P frames is formed from it. The I and P frames 
are called anchor frames. 

The B frames occur between anchor frames, and the motion vectors used in encoding them are illustrated 
in Fiq 2 The first numeral in a subscript of a motion vector mv indicates the frame in which a motion block 
is located and the second numeral indicates the frame being searched for a matching block. Thus, in 
encoding the frame P3 as noted above, mv30 is a motion vector indicating the position of a block in 10 that 
matches a motion block in frame P3 and mv63 is a motion vector indicating the position of a block in P3 
that matches a motion block in P6. 

For each motion block in a B frame, two motion vectors are derived, one referenced to a subsequent 
anchor frame and one anchor referenced to a previous anchor frame. Thus mv23 ind.cates the pos.t.on of 
a matching block in P3 relative to a motion block in B2 and mv20 indicates the position of a matching 
block in 10 relative to a motion block in B2. What to transmit to the decoder can be determined by finding 
for each block which of the following modes of operation produces the least Mean Square Error ^(MSE) as 
this will reduce the bit rate required. For each motion block, the MSE ,s calculated by adding the squares 
of the differences between the corresponding pixels of the motion block and the matching i block and 
dividing the result by the number of pixels in the motion block.ln what is called a backward mode, the MSE 
is calculated between the motion block in B2 and the matching block in P3, and in a forward mode the 
MSE calculation is done with the matching block being in 10. In what is called an interpolate mode, the 
matching blocks in P3 and 10 are averaged, and the MSE is calculated with this averaged matching block. 
Whichever mode produces the least MSE is conveyed to the decoder. Thus, .f the backward mode 
produces the least MSE, a motion vector mv23 is transmitted along with an indication that it is the 
backward vector, and if the forward mode produces the least MSE, a motion vector mv20 IS transmitted 
along with an indication that it is the forward vector. If the interpolate mode produces the least MSE, both 
vectors mv23 and mv20 are transmitted along with an indication that both vectors are being transmitted.Of 
course, for the selected mode, the residue obtained by subtracting the matching block from the motion 
block is also encoded and transmitted. 

Fig. 1 shows the various frames in the order that they are received by the encoder and prese frted to .the 
viewer and Fig. 3 illustrates the order in which the frames of Fig. 1 are processed. Both P-3 and 10 must 
be available before B-2 and B-1 can be processed as described above, and 10 must be retained in 
memory until P3 is available before B1 and B2 can be processed. The frame order of Fig. 3 is also the 
order in which the frames are transmitted. 

In summary, transmission must include the 10 frames, the P to I frame motion vectors such as mv30, the 
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residue for predicted frames such as P3, the residue for each B frame that results from the setocted mode 
S operation and a t least one motion vector for each B frame along with s.gnals indicating the mode of 
opeS that has been selected. How a decoder can reconstruct the frames from th.s information is 
thought to be obvious. 

In article 4 which is incorporated herein by reference, a differential pulse code modulation system is 
deso ribec I ihat S like the IS system described above except for the fact that the mo .on vectors for the B 
framerare derived from motion vectors such as mv30 or mv63 of Fig. 2 that ind.cate the relative position 
otabtolX ,?n anchor frame that matches a block in the next anchor frame. The motion; ,ec tor selected 
t a Son block in a B frame is the motion vector for a block in the later anchor frame Ja s n he a me 
spatial position If linear motion is assumed, a motion vector mv20 (., j) would equal 2/3 mv30 (., j) : and 
mv23 fi n would equal -1/3 mv30 (i, ]). While this method has the advantage of not transmitting motion 
Tctors forTe B frames the motion vectors that are used require the transmission of a s,gn,ficant amount 
of residue in most cases. 

In article 5 which is incorporated herein by reference, the blocks in the earlier and later anchor Barnes that 
are a the eS of a vecto? such as mv30 or mv63 that pass through a block in a B frame are averaged to 
Produce me data in that block. Although this method does not require the transm.ss.on of motion vectors 
for the B frames, it suffers from gap and overlap problems. 

BRIEF DESCRIPTION OF THE INVENTION 

In accordance with this invention, motion vectors for a block in a B frame are derived from the motion 
lect^foV a motion block in the following anchor frame whose projection along .ts mot.on vector to the 
Sous anchor frame has the most overlap with that block. Thus, it is not necessary, as on the IS system, 
fofirS making blocks in subsequent and previous anchor frames so as to find " 
mv?0 and mv23 of Fiq 4 Instead, the motion vectors for a motion block in a B2 frame are mv20 (. j) - 2/3 
Zlo ?k I) and m23 (i j) = 1/3 mv30 (k, I), where k and I are selected, as explained in the description of 
Ra 4 Once theTe B frame motion vectors are derived, they can be used at the encoder in he various 
modes dlscufsfd in connection with the IS system to determine which mode produces the least residue.U 
is then only necessary to transmit the mode rather than the motion vectors. 

Whereas the motion vector for the later anchor frame is expressed in integral numbers of pixel spacing 
Amotion vecto for a block in a B frame may be expressed in terms of fractions of pixel, spacing^ This .s 
because the motion vector for a B frame is a fraction of the motion vector for a P frame Where two B 
frames B1 and B2 are used as in the drawings, the resolution of the shorter motion vectors for B frame 
motion Socks is 1/3 of a pixel spacing and that for the longer motion vector .s 2/3 of a p.xel spacing. 
Therefore when these motiln vector! are used in reconstructing a frame from pixels and .n terpolation .s 
™5 a better image can be formed. In order to obtain an independent t mot.on vector f W,th 1/3 poce. 
resolution for a motion block in the B2 frame, nine times as many locations .n each of the 10 and P3 
frames would have to be examined. 

The major advantages of deriving the motion vectors for the B frame from the P frame vectors in 
accondance wifh this invention lie in the fact that in most situations less res.due is produced than .n the IS 
system and n^ motion vectors for the B frames need to be sent. As in an IS system, a s.gna. indicating 
mode has to be sent, but it requires significantly fewer bits than a motion vector. 

BRIEF DESCRIPTION OF THE DRAWINGS 

Fig. 1 illustrates the I, B and P frames used in a IS system; 

Fig 2 illustrates the motion vectors used in an IS system; 

Fia 3 illustrates the order in which the frames are processed in an IS system; 

Hg 4 illustrates how motion vectors for B frames are derived from motion vectors for P frames in 

rTSu^ in a B frame that can be intercepted by the projection of a motion block in 

thA P tramp alona its motion vector to a previous anchor frame; 

Fig. 6 Sows ^thelnpurand outputs of an area overlap motion vector calculator used in an embodiment of 

FwsTSTand 1 0 constitute a flow chart for the operation of an area overlap motion vector calculator of 

this invention using the upper left corner of a motion block in a P frame as a point of reference. 

Fia 1 1 is a block diagram of an encoder for a system employing independent search IS, 

Fig. 12 S a block diagram of means for selecting the mode for predicting B frames and for form.ng the 
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predicted B frame in an IS system; 

Fin 13 is a block diaqram of the decoder for an IS system; 

Fig. 14 is a block diagram of an encoder using the area overlap method of this invention for determining 

motion vectors for predicted blocks in B frames; . « * ^ < ^ tn „ 

Fig 15 is a block diagram of one means for selecting, the mode for predicting B frames and for forming 

Fig P 16^ of a second means for se,ectln 9 the mode for Predicting B frames and for 

F^Um ffiS 5fa d gfamoTa S decoder for a system using the area overlap method of this invention for 

determining motion vectors; . 

Fio 18 illustrates the order in which frames are received and processed at the decoder and 

Fig. 19 illustrates the order in which frames are output from the decoder to image reproduction means. 

DETAILED DESCRIPTION OF THE INVENTION 

Reference is made to Figs. 4 and 5 for an explanation of the area-overlap method used by this invention to 
Snve > th 'motion vecto ! for a b.ock in a B frame. In Fig. 4, frames .0, B1 B2 and I P3 are sh own u They are 
Soned into identical rectangular blocks of pixels that are L1 pixels w.de and L2 p.xels h.gh and located 
Kcordance with i and j coordinates having an origin at the upper left corners of the frames. With.n each 
frame, pixels are identified by x, y coordinates. 

In Fiq 4 a motion block at i = 0, j = 0 of a P3 frame is shown by the shading as being promoted toward the 
u frame in a 6 ection parallel to ts motion vector 2. The area of intersection of the projection wrth the B1 
framed shown by a rectangle 4, and the area of intersection of the projection with the B2 frame is shown 
by a rectangle 5. 

A method for determining the area of overlap of the area of intersection 4 with the various I ^ksoftoe B1 
frame will now be explained by reference to Figs. 4 and 5. Although any point in a motion block of P3 
could be used a? a reference, the upper left hand corner 6 is used in this particular example. Its projection 
fe shown as^mersecting the B1 frame at 6' in a block at j = 0, i = 2. Observation will show that the area of 
intersection 4 can only overlap four blocks in B1 , the one intersected by the projection of the corners _ 
which is 6\ the one to the right, the one below and the one on a diagonal, i.e. those at j - 0, i - 2, j - i , i - 
2, j = 0, i = 3 and j = 1, i = 3. 

The area in pixels of the overlap with each of the four blocks can be determined by the following formula: 

V^/^JS^Si^SEP* - u<SEP> - L1 <SEP>y - v<SEP> 
where L1 equals the width of a motion block in pixels, L2 the height of a motion block m p.xels. u, v are the 
values of the x y coordinates in pixels of a reference point having a defined relationship to the projection 
ofa motion block ?n She P frame onto a B frame, and x. y are the coordinates in pixels of a reference po.nt 
having the same defined relationship to a B frame motion block. 

Bv wav of example suppose we wish to find the area of overlap of the projection of the motion block in the 
uDDerlefl £S2 of the P3 frame of Fig. 4 with the motion block at j = 1, i = 2 of the B1 frame. As shown in 
upper left corner or me = y observatjf)n me wkJth of the over|ap is 12 pixels and lts hei ght 

is 6 so that he desired area is 12 x 6 = 72 pixels.lf the defined relationship is that the reference point is 
Ihe top left comer of a block, then u = 1 2 and v = 18 and x = 1 6 and y = 1 6 so that the area of overlap .n 
accordance with the formula is: 

AO = [16 - <SEP>16 - 12<SEP>] [8 - <SEP>16 - 18<SEP>] 
= [16 -4] [8 -2] 

As illustrated by the flow charts of Figs. 6 through 10 for calculating the areas of overlap AO the > areas of 
ove lap in each of the blocks of a B frame are derived for each motion block m a P frame and the vector 
for the block in the B frame is derived from the vector of the motion block in the P frame whose projection 
has the most overlap. 

Reference is now made to Figs. 6 through 10 for a description of a flow chart of a way for determining the 
motion vectors of a B frame in accordance with the algorithm just descnbed. It is thought that the 
comments in the various blocks make the chart self-explanatory, but its general operation is as follows. 
The following terms are used: 
T - frame number; 

N - (N-1 ) B frames between anchor frames; 

Area Table - current best area overlap for each B frame motion block; 



http://v3.espacenet.com/textdes?DB=EPODOC&IDX=EP0534350&F=0&QPN=EP0534... 6/16/05 



esp@cenet description view 



Page 4 of 8 



Best mv - current best P frame motion vector for each B frame motion block; 

t if-tSn oTp Inj'SlSSck, From .op ,eft come,. , mofon Pfocks down, j motion bloofcs 

PpiTiS pi" (B3 b SrPTB'4Tkica.ion of B frame mofion bfocks.From fop left comer iBk 

motion blocks down, jBk motion blocks to the right; 

(x v) - Pixel coordinates of top left corner of B frame motion blocks; 

(tlvP tlhP) - pixel coordinates of top left corner of P-frame motion block in the P-frame; 

(u, v) - pixel coordinates of top left corner of P frame motion block projected onto B frame; 

k - B frame motion block loop counter; and 

AO - area overlap. 

m a block 7 of Fiq 6 the motion vectors (PMv) from the next anchor frame to the previous anchor frame 
are supped to the calculator, and it supplies, in a manner explained in connection w.th F,gs. 7 through 10, 
the forward and backward motion vectors for a B frame. 

The blocks 8 10 12 and 14 of Fig. 7 are for initialization. In block 16, the pixel coordinates of the top left 
Srne of a P frame motion block are found, and using the P frame motion vectors block 1 8 ^locates ; the 
coord nates of the top left corner of the projection block onto the B frame of interest. In block 20 the block 
fn toe B frame trough which the projection of the upper left corner of the motion block in he F» frame 
□asses and in block 22 he other three blocks where there may be overlap wrth the projected 

motS ^locl c to the P frame are identified. In block 24, k is set to 1 and the next t,me through the loop .t w,ll 
be set at 2. 

In block 26 the x and y coordinates in pixels are determined for each of the blocks identified in blocks 20 
22 and 28 and toe foLula for AO is applied to each of them. As the AO for each of the four blocks in a B 
fram is calculated a decision is made, block 30, as to whether the AO is greater than an AO Prev'ous.y 
calculated If it is larger a block 32 updates a table for that B block, and the procedure goes to a block 34 
which adds to k IfWock 30 indicates that toe AO is not larger than what is in the table the table is not 
changed and toe procedure goes to the block 34. A block 36 asks whether k is greater than 4. If not, then 
the procedure loops back to the block 26 as indicated by a line 37. 

If k is creator than 4 it means that all four blocks which a projected motion block of a P ' frame might 
ntersSf have been processed. A block 38 then iterates i and j so as go to another motion block ^,n the P 
frame A bfock 40 then checks to see if all P frame motion blocks have been processed. » not the 
procedure loops back to the block 16 as indicated by the line 41 . If so, . and j are set to O in block 42 and 
toTprocSure il set to derive the forward and backward B frame motion vectors from toe selected P frame 
motion vector. 

Then for each B frame motion block, the vectors from the table are obtained that are for the motion block 
EaP Sme having projection on that B frame block with the most overlap, block 44. The vectors are 
hen sSed so as to take into account the position of the B frame between two anchor frames. If B1 is the 
btock u^der consideration, the backward vector would be -2/3 mv30 and the J^* 1 * 
mv30 As this is done for each B frame block, the i and j coordinates are .terated A block 48 asks al B 
frame blocks have been processed. If not, the procedure loops back to toe block 44 as indicated by a line 
45 bJ f! S B Se blocks have been processed, the calculations are complete unt.1 ,t .s des.red to obtam 
the motion vectors for the blocks of another B frame. 

Fiq 1 1 shows a block diagram of an encoder for a system which uses the independent search method 
p eviousiy mfened to in which forward and backward vectors are derived for each block in a B frame by a 
btock matching process. Signals from a source 51 are digitized in an A/D device 52 and applied to the I P 
terminal ofa Switch 53. When coding an I frame, the switch 53 is in contact with its I P terminal so that the 

for this frame are transferred block by block to the positive input of a subtracte 54. Also the 
nSveinpuUs connected to ground by a switch 61 . The blocks of digit.zed data are transformed block 
KSSStoDCT coefficients by a Discrete Cosine Transform processor 55. These ooeffloante are 
quantized by a quantizer 56 so as to reduce the resolution of coefficents for h.gher frequences. 

The quantized coefficients are variable length coded by a variable length coder, VLC, 63, and the resulting 
bU stream fs Coupled to a FIFO buffer 64. The output of the FIFO buffer 64 .s a bit stoaam having a 
constant b! ; rate and is transmitted to an output channel by a signal conveyor 66. The fullness of the FIFO 
Se 64 is Examined by a buffer controller 65 that adjusts the step size of the quantizer 56 so as to 
tocrease or decrease the bit rate of the signal applied to the VLC 63 and thereby prevent it from 
lowing Thus, an I frame is intra-coded so that no reference is made to any other frame^ The I frame 
^ recovered albe t with quantization error, by an inverse quantizer 57 and .nverse DOT IDCT 58 and 
applS Z an adder 59 At this time, the output of the adder 59 is connected v.a a switch 120 to the input of 
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a series of frame delays 61 a - 61 e. 

Note that all digitized frames at the output of the A/D device 52 are applied to a series of frame delays 60a 
throuqh 60e. The B frames immediately following the I frame just described are applied to the frame 
delays 60a - 60e. but the switch 53 is input connected to terminal B so that the B frames are merely held 
for future processing, while previous B frames are supplied to terminal B of switch 53 for processing. 

The next P frame, P3, for example, is encoded by again placing the switch 53 in contact with its I P 
terminal and placing the switch 61 in contact with its P, B terminal. The switch 120 at the output of the 
adder 59 is connected to a terminal 121 that is connected to the input of a series of frame delays 62a - 
62e. 

The input digitized 10 frame is now at the output of the frame delay 60c which is connected to the f input of 
a motion estimator 67, and the input P3 frame is applied to the g input of the motion estimator 67. In .a 
manner well known to those skilled in the art, the motion estimator 67 derives the motion vectors like mv30 
of Fig 2 that indicate for each motion block in P3 where a matching block may be found in 10 '-These 
motion vectors appear at the output FMv of the motion estimator 67 and are applied to the VLC 63 for 
transmission to a decoder and to a motion compensator means 68. 

The motion compensator 68 is coupled by a line 68' to the positive input of the adder 54 so as to receive 
the diqitized input data for P3, and a switch 73 connects an input of the compensator 68 to the output of 
the delay 61c so as to receive the frame 10 that was recovered by the inverse quantizer 57 and the inverse 
OCT 59 Bv usinq the motion vectors from the motion estimator 67, the motion compensator 68 forms a 
predicted frame for P3 which is supplied to the adder 59 and via a line B and the switch 61 to the negative 
input of the subtracter 54 wherein it is subtracted from the input data for P3 by the subtracter 54 so as to 
derive the difference or residue signal. 

This residue signal is transformed into DCT coefficients that are quantized in a similar way as the 10 frame 
mentioned earlier. Also, the operations performed by the VLC 63, the FIFO 64, the inverse quantizer 57 
and the inverse DCT 58 are the same as when the I frame was processed. The output of the IDCT 58 
represents residue and is added to the predicted P3 frame by the adder 59, and the adder output is stored 
in a series of frame delays 62a - 62e. Note that the processed anchor frames are alternately stored in the 
frame delays 61a - 61e and 62a - 62e. Thus, 10 was applied to the delays 61a - 61e. 

At this point the encoder has in memory the original data for the 10 frame and a P3 frame that has been 
derived from the 10 frame in the manner described. At a decoder, the coded 10 frame is decoded and a 
predicted P frame is derived from it with the aid of the motion vectors like mv30. The coded residue signal 
is decoded and added to the predicted P frame to produce the P3 frame. 

In an IS system, the best mode for forming a B frame from the I and P frame on either side of it is 
determined in the following way. The input B1 frame is now available at the output of the frame delay 60c 
that is connected to the f input of the motion estimator 67. 10 is at the output of the frame delay 60d and is 
connected to the input h of the motion estimator by positioning a switch 77, and the input frame P3, which 
is at the output of the frame delay 60a, is coupled to the input e of the motion estimator 67 by positioning a 
switch 76 For every motion block in B1 , the motion estimator 67 finds a matching block in the previous 
anchor frame 10 so as to produce a forward motion vector FMv such as mv10 of Fig. 2. For every motion 
block in B1 a matching block is also found in the next anchor frame P3. This results in a backward motion 
vector BMv such as mv13 in Fig. 2. 

One or both of the motion vectors derived as just described may be sent to the VLC 63 for coding based 
on a decision made by the motion compensator 68 in a manner illustrated in Fig. 12. The selection made 
is called a mode, and a signal indicating the mode is sent via a line M to the VLC 63 for encoding. The B 
frames are not transmitted, but they are reconstructed at a decoder from one or both of the anchor frames 
bv using the motion vectors in accordance with the mode signal. Thus, if the mode signa indicates that the 
matching block in P3 gives best results, that matching block is used. If the matching block in 10 is best, .t is 
used, and if a combination of blocks produces the best results, the combination is used. This in done on a 
block by block basis so that different modes may apply to different blocks. 

In addition to motion vectors and mode signals, the encoder must encode the difference between a 
selected matching block or combination of matching blocks and the corresponding block in the B frame as 
it is supplied by the A/D device 52. This is done by supplying the selected matching block or combination 
of matching blocks to the negative input of the subtracter 54 via the line B and the switch 61 The 
differences or residue at the output of the subtracter 54 are then processed by the DCT 55, the ^niizer 
56 the VLC 63 the FIFO 64 and applied to the signal conveyor 66. There is no need to form a predicted 
block with the inverse quantizer 57 and the inverse DCT 58 as decoded B frames are not needed by the 
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encoder and therefore need not be formed at the encoder. Therefore, the switch 120 is set on an open 
terminal. 

The manner in which the best mode is selected in an IS system is as follows ^Whereas the ^motiorv vectors 
FMv and BMv for a B frame are derived from the actual anchor frames on either side of it, these >actuBl 
frames are not available at the decoder so that the best mode must be selected on the basis of the anchor 
frames available at the encoder. Anchor frames substantially identical to those recovered at a decoder 
were oroduced at the output of adder 59 and alternatively applied by the sw,tch 120 to the senes of frame 
delays 6 1 61 e and the frame delays 62a - 62e as previously described^ Future .anchor frames are 
supplied to an input a of the motion compensator 68 by means of a sw.tch 69 and switches 71 and 72. and 
nast anchor frames by switches 70, 74 and 75 to an input b of the motion compensator 68 By way of 
examp"?whenTlS ^ best mode is to be determined for B1, the switch 71 is placed in contact with i the output 
5S?S5™^«Srand the switch 69 is placed in contact with the switch 71 so that the ,n«d anchor 
frame P3 which is at the output of 62a is applied to the a input of the motion compensator 68, the switch 
74r D laced in contact with the output of the frame delay 61 d. and the switch 70 is placed in contact with 
[he sw to ^4 sSThe previous anchor frame .0, which is at the output of the frame delay -61 c .s applied 
to thTb input. Also, switches 77 and 76 are appropriately set so as to choose mput frames 10 and P3 
respectively. 

Referrinq to Fig 12, the FMv and BMv motion vectors supplied by the motion estimator 67 are used to 
sefec the appropriate forward and backward predicted motion blocks from the past and future anchor 
frames by s5on blocks 80 and 89 respectively. These motion blocks are res pec ively ^supphed to Mean 
Square Error (MSE) calculators 85 and 84. The other input to each of these calculators is the input B 
frame mo on block Additionally, a third motion block called an "interpolative predicted motion block may 
be genera tod by performing a weighted average on the forward and backward predated motion blocks. 
For Ixampfe an amplifier 81 may multiply the output of the block from the past anchor frame by K and the 
one from toe futore anchor frame by 1 -K. In the case of B1 . K = 2/3. The interpolative predicted motion 
block is supplied to a third MSE calculator stage 86. 

Each of these MSE calculators calculates the mean square error between the input motion block and the 
oredicted or matching motion block These errors are supplied to a selector 87 which determines the 
minimum o >SS££ errors and controls a switch 88 so as to choose the P^ te ^^ p ^LT 
resulted in the least error This chosen predicted motion block appears on the lead B of the motion 
SSS^C^R^i and is supplied through the switch 61 to subtracter 

as to which predicted block is selected is supplied by the selector 87 and by a lead M to the VLC 63 for 
coding and transmission to the receiver. Specifically, the mode information tells the receiver wbnch .of the 
toVe^prediction modes was used in the prediction of each motion block when frame B1 was coded. 

Figure 13 shows the block diagram of the decoder for the independent search method 'S-^*"*^"* 
97 receives the bit rate reduced signal and supplies . to a variable length , decoder ^D) QS^The VLD 
separates the block data into essentially four types of information: video data in the form of quantized DO I 
coeffidents forward motion vectors FMv, backward motion vectors BMv and mode ■*gg*^he 
quantized DCT coefficients are dequantized by inverse quantizer 99 and supplied to a , IDCT J0C which 
tofnsforms them into residue pixels to be added to the prediction L ,nf ° r t ^ a £°" ^V^l^ oixt.l? 
following manner. Note that the prediction information is absent for m ^^j^?^^^ 
orediction for P frames is obtained as follows. One of the two frame stores 102 and 103 contains the 
oreviouranchor frame which is to be used in generating the prediction. Using the forward motion vector 
SSZ^TK a^Snrti^ block is chosen from the previous anchor frame. After passing through 
Sch 107 it is supplied to an adder 101 where it is added to the residue signal to form the P frame .at a 
Sal 109. Just as in the encoder, a switch 106 is operated so as to store received anchor frames 
alternately in frame stores I and II, but received B frames are not stored in either frame store. 

When B frames are to be predicted, the mode information instructs the displacement blocks 104 and 105 
Ts well as switch 107 as to the type of prediction to be used (i.e. forward, backward, or interpolative) 
Using this So^atfon as well asThe appropriate motion ^ 1 ^^^^ d ^ZT' 
the proper motion block or blocks are chosen from one or both frame stores 102 and 102k Erther the 
forward or backward or interpolative predicted motion block .s supplied through the switch 107 to .the 
SddJlOT. An adder 108 performs the weighted average of the forward and backward motion blocks as 
required for interpolative prediction. Frame stores I and II each have taps for a delay of 1, 2, 3, 4 or 1 5 
frames in a ^manner simitar to frame delays 61 and 62 in the encoder.The predict-onis passed to the adder 
101 where it is added to the residue signal to form the reconstructed B frames at the terminal 109. 

The reconstructed frames will appear at terminal 1 09 in decoder processing order Fig. 1 B. Switch 1 30 
puts toe ack toto display order as shown in Fig. 19. When a reconstructed B frame occurs at 
tomdrari 109 Sitoh 130 is set so as to supply it immediately to video output terminal 131 . When 
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reconstructed anchor frame occurs at terminal 109, switch 106 is set so as to route Jtoj-J^jj 18 
stores; and switch 130 is set as to supply the anchor frame .n the other frame store to the v.deo output 
terminal 131. 

K to b 6 IrSSd is P6. It is stored in frame stere I while P3 in freme store II rs sent out. 



Area Overlap Method (AO) 



The Area Overlap method of this invention differs from the Independent Search method ma.n'y .n the > way 
thai thel^edfction of B frames is done. The AO method uses P frame motion vectors such a i mv30 for 
pred K B Sme motion blocks in the following way. In this discussion reference »s made to the use of 
motion vectors for frame P3 (with respect to 10) for the purpose of pred.ct.ng B1 and B2. 

Fia 14 is a block diagram for an encoder incorporating this invention. Those components corresponding to 
component Tin R 11 are designated in the same way. Forward motion vectors for P3 such as mv30, are 
S n memorv 89 for a period (delay) of one or two frames as requ.red by frames B1 and B2 
f£oe2l\l r ( se?Rg 3) -I?e stored motion vectors are supplied by the Fmv Memory 89 to an Area 



lead M\ 



Motion compensetion may be done in a number of ways. One way is shown in F 1g. 1ft » 

the, rsntnut nf thp adder This becomes a fourth candidate to be chosen by the switch 88. The calculations 
JriSSl^iS^Ji^ an MSE calcu.ator 96. and the results are supplied to m'n.mum sele ctor 
8?Tn rmanneTpreviously explained in connection with Fig. 12.We now have four possible predicted 
mo ?nn Socks anTonce again the selector 87 instructs switch 88 to select the predicted motion b ock 
which ^ as to which of these modes yields the least .error f .s also sent 

iJth e receiver as before Fig 16 is an alternative method for motion compensation. It d.ffers from Fig. 1 5 
on vln She AO predicted motion block has replaced the interpolate predicted motion block as a 
carafe whereas' in Figure 16 the AO predicted motion block was added to the list of cand,dates to be 



checked. 



Whereas the AO motion vectors, AO FMv and AO BMv, derived in accordance with this invention have 
teen [shown as b™g usS in a bidirectional mode, they may also be used in unidirectional modes in a 
manner similar to the FMv and BMv vectors of the IS system. 

A decoder for the AO method is shown in Figure 17 in which components corresponding to those of Fig. 
13 ; Ire ^desiqnated *n the same way. In view of the fact that motion vectors for B frames can also be 
derived I in Srdance with this invention from motion vectors for an anchor frame that is ; referenced to ; a 
^SSTJSSXSSL. blocks 1 10 and 1 1 1 are used by the decoder ^^SS^^JS^^ 
to blocks 89 and 90 in Fig. 14. The mode information .nstructs the displacement blocks 104 and I 105 as 
we?, as Lrtch 10fas to the prediction mode to be used for B frame motion blocks. Otherwise, the 
operation of this decoder is identical to that of the decoder for the IS method. 
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Derivation and use of motion vectors in a differential pulse code modulation 
system 

Claims of EP0534350 

'^^S^n^SS&^s , and predicted frames P, said anchor frames 
havina a plurality of B frames between them; kl 

r^««X^on K o. ,ha proiecon a. each ration block in an anchor frame along 

rntSrrSl1io?S,r.or a 0,0* ,n a B frame from me motion vecfo, o, a black ,n an anchor 
frame whose projection overlaps it the most. 

m ^~an"r mSglnfraframes , and prediCed frames P. said anchor frames 

means for providing motion vectors for each block in an anchor frame that identify the position of a 

S^^'ff^^SSSST- the projection o. each mo«an b,oc k in an anchor frame aiang 

meanstr SS», 'Stock in a B frame the black in an anchor frame whose area o, in.ersac.ion 

ESSE," SvTnTmoSn veCora for each block in a B frame from the motion vector far the laa. 
mentioned block in an anchor frame. 
Anoaratus as set forth in claim 2 wherein said means for defining said area of intersection .comprises. 

S"ate tXean^tid B frame and said anchor frame and the denominator of said fraction being the 
ir^r^St-«lp to sa* second point as the b,ack in 
said anchor frame has to said first point. 

^Apparatus as set forth in claim 3 wherein said reference point is at a comer of said block in said anchor 
frame. 

s Anoaratus as set forth in claim 3 wherein an area of intersection of the projection of a block in said 

U fste dimension of a block as measured along a first direction from said given corner that is parallel to 
Sit? dimen2on e of a block as measured along a second direction from said given corner that is 
fandTar^the coo^slSg said first and second directions respectively of a point having a given 

relatfo^ to the intersection of the projection of said block .n sa,d anchor frame. 

c* A n onrnHpr for a differential pulse code modulation system comprising: 

mean! SSg Si for motion blocks of an anchor frame that are referenced to a prevous 
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m n ea h ns £ deriving first sets of forward and backward motion vectors for motion blocks of B frames that 
are ^speS^eLced to the next and previous anchor frames, the first sets of mot.on vectors be.ng 

^S^g'S^SZS^ and backward motion vectors for b.ocks in B frames that are 
Soect^e^ to the next and previous anchor frames, the second sets of mot.on vectors .be.ng 
SS from the motion vectors for the motion blocks of an anchor frame that are referenced to the 

^S^l^^Se^ comparisons of blocks in anchor frames that are indicated for each motion 
block of a B^frame by the forward motion vectors of the first sets, the backward mofion vectors of the first 
sets and the averages of the blocks so indicated with the input blocks of the B frame so as to denve 

meTn?^ * anchor frames that are indicated for each motion b.ock 

of E . frame byTI forVard and backward motion vectors of the second sets so as to denve represents 

mean XX^Z^ for each motion b.ock of a B frame indicating the set of motion vectors 
containing the motion vector associated with the smallest difference. 

7 An Pncoder as set forth in claim 6, wherein said means for deriving second sets of forward and 
backward m^tfon vecTors for b.ocks'in B frames that are respectively referenced to the next and previous 

metns for partSg'th? anchor frames and B frames into a plurality of identical blocks; 

means for defining the area of intersection of the projection of each mot-on block .n an anchor frame along 

Tans SSS^^E^ in a B frame the b.ock in an anchor frame whose area of intersection 

™£ns*o! S^SL vectors for each b.ock in a B frame from the motion vector for the <ast 
mentioned block in an anchor frame. 

8 A decoder for a differential pulse code modulation system comprising: 
means for deriving successive anchor frames from received encoded signals; 

means for and backward motion vectors for the motion blocks of B frames; 

means for receiving motion vectors for motion blocks in an anchor frame that are referenced to the 

SSS?^L?SS. said .ast mentioned motion vectors second sets of forward and backward motion 

^SS^SSSSfS a B ^Sedframe from the blocks of anchor frames in accordance with 
motion vectors from a set of motion vectors that is indicated by a mode signal. 

9 A decoder as set forth in claim 8 wherein said means for deriving said second sets of forward and 
backward motion vectors for the motion blocks of a B frame comprises: 

mtans for Dartitioninq the anchor frames and B frames into a plurality of identical blocks; 

means for defining [he area of intersection of the projection of each.motion block in an anchor frame along 

in a B frame the b.ock in an anchor frame whose area of intersection 

m^anTfo? J2r5?nSSn vectors for each b.ock in a B frame from the motion vector for the last 
mentioned block in an anchor frame. 
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